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Abstract: There has been increasing interest in an evidence-based approach to 
education in Australia, but relatively little research has provided relevant data on 
knowledge of the evidence base for instructional practices among teachers 
preparing to enter the profession. Final year teacher education students (N = 

290) in 15 Australian tertiary institutions were surveyed on their understanding 
of the strength of evidence for 14 instructional strategies and their intended 
frequency of use of the strategies following graduation. They were also asked to 
rate the importance of factors they considered in instructional decision-making. 
Empirical evidence was importan t in selection of instructional practices but 
personal preference and, in particular, practicum experiences were considered 
more important. Students were very confident in their ability to make judgements 
regarding the evidence base for a range of instructional strategies and tended to 
rate all strategies as relatively effective. Their judgements, however, did not 
correlate strongly with available evidence. Intended use of strategies correlated 
highly with strength of evidence ratings. Implications of these findings for teacher 
preparation and future research are considered. 

Keywords: evidence-based practice, teacher education students, instructional practices. 


Introduction 

The presence of a research to practice gap in education is widely asserted (e.g., 
Everett, Luera, & Otto, 2008; Gable, Tonelson, Sheth, Wilson, & Park, 2012; Jones, 2009) 
and in this context, there has been increasing interest in the concept of evidence-based 
practice in education in Australia (Centre for Education Statistics and Evaluation, 2013; 
Rowe, 2005; Stephenson, Carter, & O'Neill, 2013). The Australian Institute for Teaching and 
School Leadership (2011), which accredits teacher education courses in Australian tertiary 
institutions, is committed to building on research evidence about “what works in teacher 
education” (p. 3) and has noted that teacher education programs should take account of 
“authoritative educational research findings” (p. 12). Ingvarson and Rowe (2008) have 
argued that teachers are the most valuable resources available to schools, and the goal of 
raising the quality of teaching “can only be realised by ensuring that teachers are equipped 
with subject-matter knowledge and an evidence- and standards-based repertoire of 
pedagogical skills that are demonstrably effective in meeting the developmental and learning 
needs of all students” (p.6). 
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Factors Considered in Instructional Decision-Making 

The extent to which practicing teachers rely on research in their instructional 
decision-making has been examined from several perspectives. A review by Rudland and 
Kemp (2004) provided examination of the professional reading habits of teachers. They 
concluded that teachers engaged in relatively little professional reading in comparison with 
other professions. Further, the reading of teachers tended to be of a pragmatic nature rather 
than research focused. In light of this finding, an obvious question would be what factors 
teachers consider in instructional decision-making. Landrum, Cook, Tankersley, and 
Fitzgerald (2002) provide some insight into this issue. They examined the views of student 
teachers nearing the end of their courses in two North American universities. Landrum et al. 
reported that respondents considered the opinions of colleagues, workshops and in-service 
programs were more accessible, usable and trustworthy than professional journals. An 
interesting study by Foegen, Espin, Allinder, and Markell (2001) provided evidence that pre¬ 
service teachers’ views on the validity and utility of curriculum-based measurement, a 
specific type of formative evaluation, was not increased by presentation of statistical 
information, as compared with anecdotal first-person accounts. Thus, there is some evidence 
to suggest that both practicing teachers and those in preparation may not regard research as 
fundamentally relevant to instructional decision-making. 


Knowledge of Evidence-Based Practices 

There are a number of studies that have examined working teachers’ knowledge of 
evidence-based practices but these have tended to focus on narrow student groups, very 
restricted geographical areas or specific professional groups. For example, Gable, Tonelson, 
Sheth, Wilson, and Park (2012) examined rated importance, reported use and level of 
preparation in practicing North American teachers with respect to 20 evidence-based 
practices for students with emotional disabilities. A number of evidence-based practices were 
not in common use and teachers indicated a lack of adequate preparation on important 
strategies, including function-based interventions. Gable et al. (2012) concluded that there 
appeared to be a substantial research to practice gap for both special education teachers and 
general education teachers working with children with emotional disabilities. Similarly, 
Stormont, Reinke, and Herman (2011) surveyed 239 early childhood and elementary regular 
education teachers from five US school districts on 10 intervention approaches with 
extensive empirical support in addressing the needs of students with emotional and 
behavioural problems. They found that over 80% of teachers had not heard of nine 
approaches and 10% or less of teachers agreed or strongly agreed that the approaches were 
evidence-based. Burns and Ysseldyke (2009) and Carter, Strnadova, and Stephenson (2012) 
examined the reported level of implementation of eight instructional practices by special 
education teachers in the United States and Czech Republic respectively. In both studies there 
were moderate to high levels of reported use of all practices, including those with little 
empirical support that could not be considered evidence-based. 

In addition to this international research, there have been a number of Australian 
studies examining teacher knowledge or implementation of evidence-based practices. In a 
replication of the previously mentioned study of Bums and Ysseldyke (2009), Carter, 
Stephenson, and Strnadova (2011) examined the reported use of evidence and non-evidence- 
based instructional practices by special educators, with broadly similar findings. Demant and 
Yates (2003) examined knowledge and attitudes of 58 Australian primary teachers to the 
direct instruction construct, which has a strong research base. They found that teachers 
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tended to have generally positive attitudes, which tended to correlate with more accurate 
knowledge. 

Thus, existing research with practicing teachers has tended to focus on specific 
professional groups (such as special education teachers) or practices related to particular 
student groups (such as those with emotional difficulties). In addition, researchers have 
tended to focus on issues such as reported familiarity with procedures or reported 
implementation. Largely missing from these data has been examination of teacher knowledge 
of the research base for these procedures. Given that the interest in evidence-based practice, 
both internationally and in Australia, has been relatively recent, it is certainly possible that 
findings from practicing teachers may not hold for students currently in teacher preparation 
programs, particularly those approaching graduation. Given recent emphasis on the need for 
Australian teacher education programs to incorporate a more extensive understanding of 
research into effective pedagogy (Australian Institute for Teaching and School Leadership, 
2011; Rowe, 2005; Ingvarson & Rowe, 2008), an increasing focus on the evidence base 
supporting common instructional practices might be anticipated. 

There has only been limited research conducted with student teachers relevant to their 
knowledge of evidence-based practices. Bain, Brown, and Jordan (2009) examined the 
accuracy of beliefs regarding childhood interventions of teacher candidates (n=351) who 
were at various levels in training. Participants were asked about specific (a) evidence-based 
practices, (b) controversial practices, and (c) practices that lacked evidence or that evidence 
demonstrated were ineffective. The practices were relevant to children with autism, ADHD 
or dyslexia, and an explanatory statement was provided with each intervention. With regard 
to effectiveness beliefs, Bain et al. (2009) reported that evidence-based interventions tended 
to be endorsed at high levels, but this was also true of some non-evidence-based 
interventions. Bain et al. (2009) noted that “endorsement rates across interventions varied but 
not in a consistently logical manner” (p. 85). All non-evidence-based interventions were 
endorsed as effective by more than a quarter of respondents (including some that are 
dangerous, such as vaccine withdrawal and chelation therapy for autism). It was also notable 
that many respondents tended to endorse interventions as effective, even though they 
acknowledged that they had not had previous knowledge or exposure. There was no 
consistent evidence of improvement in judgement as trainees move closer to licensure. It 
should be noted that this study was limited to a single North American institution and some 
of the interventions examined were not educational in nature. 

In an Australian study of pre-service teachers, primarily focussing on issues of self- 
efficacy, Main and Hammond (2008) noted that teachers had limited knowledge of several 
important empirically verified behaviour management strategies, including functional 
behavioural assessment. Similarly, O'Neill and Stephenson (2011, 2012a, 2014) reported that 
the classroom and behaviour management content of pre-service primary teacher education 
programs often included superficial coverage of a large number of theoretical models and 
lacked coverage of evidence-based models. The teaching of reading in Australia has also 
been examined in relation to teacher knowledge and implementation of research-based 
practice in early literacy and questions have been raised regarding the level of knowledge of 
pre-service and beginning primary teachers (Louden & Rohl, 2006; Mahar & Richdale, 

2008). Nevertheless, as far as can be ascertained, there are no Australian studies that explore 
the knowledge of research support for broad a range of instructional practices with pre¬ 
service teachers. Although there are many survey studies of pre-service teachers, there are 
relatively few that have surveyed students across multiple universities. 

In summary, existing research has typically focussed on awareness and reported 
implementation of interventions with circumscribed groups of practicing teachers in limited 
geographical areas. Given the emergence of evidence-based practice in recent years, the 
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relevant findings may not hold for teachers who are currently in preparation. Research on 
student teacher views on evidence-based practice is limited. In particular, there appears to be 
very limited data on teacher or student teacher knowledge of the research base for 
instructional practices. The main purpose of the current study was to examine the views of 
final year Australian pre-service trainee teachers regarding the level of research support for a 
variety of instructional practices. The specific research questions were: 

1. How do final year student teachers rate factors (research evidence, personal style, 
teacher preparation programs, advice from teachers, practicum experience) as 
influencing decisions regarding instructional practices? 

2. How do final year student teachers rate the research evidence for effectiveness of 
instructional practices with varying levels of empirical support? 

3. At what frequency do final year student teachers anticipate using instructional 
practices with varying levels of empirical support? 

4. What is the relationship between student teacher ratings of research support and 
empirical evidence for common instructional practices? 

5. What is the relationship between ratings of effectiveness and anticipated frequency of 
use of common instructional practices? 


Method 

Survey 


A three-part online survey was constructed and made available via the SurveyMonkey 
platform. Completion of all relevant responses was mandatory before moving from one page 
of the survey to the next. The first part of the survey provided demographic background and 
included questions addressing: gender; the level of program in which the student was 
currently enrolled (bachelor, graduate diploma, masters, other); the area of teaching (primary, 
secondary, special education, other); the area of secondary teaching if relevant; how long 
before the student completed their course (less than six months, between six months and one 
year, more than one year); and whether the current course was the first tertiary qualification. 
The question regarding the length of time before course completion was included to verify 
that respondents were eligible for the survey (i.e., in the final year of their program of study). 

The second part of the survey consisted of seven questions. The first asked “How 
important do you think it is to use instructional practices that have been shown by empirical 
research to be effective? In this context, the term empirical research involves testing 
effectiveness of practices using experiments in which student performance is measured.” The 
remaining six questions asked about the importance of varying factors (personal philosophy 
and style, advice of other teachers, research, content of education course, practicum 
experience) in deciding on which instructional practices teachers would use in the classroom. 
The two items related to research were to allow differentiation between empirical research 
and educational research more generally. In all cases, students were asked to respond on a 
five-point Likert type scale with responses ranging from very important to very unimportant. 

In the final part of the survey students were presented with a list of 14 instructional 
practices along with a brief definition of each. In relation to each practice, students were 
asked to indicate “(a) How strong do you think the research evidence is for this instructional 
practice? If you are uncertain, click the “unsure” option.” Students responded on a five-point 
Likert-type scale with options ranging from very strong to very weak. In addition, an 
“ unsure ” option was provided to accommodate respondents who did not consider that they 
had sufficient knowledge of the instructional practice to make a judgement. Lurther, students 
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were asked to indicate “(b) How frequently do you anticipate you will use this practice in 
your teaching when you graduate? If you are uncertain, click the unsure option.” In relation 
to frequency of use the following statement was added “Some of the practices may not be 
relevant to your area of teaching so you should rate these as Never for frequency.” Students 
were presented with ordinal response options {very frequently, frequently, sometimes, 
infrequently, never ) with the additional option of unsure. 

Thirteen of the practices were selected from the synthesis of meta-analyses relevant to 
educational achievement conducted by Hattie (2009) that provided evaluation of 
approximately 800 meta-analyses, 52,637 studies and approximately 236 million students. 
Hattie (2009) argued that provision of teaching and maturation affects could account for 
effect sizes of up to approximately 0.40 and that this should be regarded as a “hinge point” 
forjudging the effectiveness of educational interventions. Eight strategies that could be 
clearly defined with effect sizes well above this hinge point (0.55 - 0.90) were selected as 
evidence-based practices. Conversely, six strategies with effect sizes near or below the hinge 
point (0.06 - 0.41) were considered as non-evidence-based practices. In each case, the 
definition of the practice was based on the description provided by Hattie (2009). A list of the 
strategies selected, definitions presented and the effect sizes reported by Hattie (2009) is 
presented in Table 1. Although learning styles instruction (0.41) was close to the hinge point, 
Hattie (2009) was critical of procedures and interpretation in many of the meta-analyses. 
Further, several other reviewers (Kavale & Forness, 1999; Landrum & McDuffie, 2010; 
Pashler, McDaniel, Rohrer, & Bjork, 2008) have failed to find convincing evidence to 
support the use of learning styles in education and consequently it was classified as a non 
evidence-based practice for the purposes of this study. In addition, the concept of multiple 
intelligences (Gardner, 1993) appears to be widely discussed in education but was not 
examined by Hattie. The theory has been heavily criticized for its lack of supporting 
empirical evidence and problematic interpretation in terms of educational practice (Dekker, 
Lee, Howard-Jones, & Jolles, 2012; Klein, 1997; Waterhouse, 2006a; Waterhouse, 2006b). 
Thus, multiple intelligences was included as a potential instructional strategy that lacks an 
adequate evidence base. 
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Intervention 

Class 

Definition 

ES 

Multiple 

Intelligences 

Non 

evidence- 

based 

Teaching that considers and accommodates multiple intelligences 
(i.e., musical, bodily- kinaesthetic, logical-mathematical, 
linguistic, spatial, interpersonal, intrapersonal, naturalistic). 

NA 

Inquiry-Based 

Teaching 

Non 

evidence- 

based 

Involves developing challenging situations where students 
observe and question, postulate explanations, devise and conduct 
experiments, analyse data, draw conclusions and build models. 
Tasks are open-ended and there is no single “right” answer. 

0.33 

Meta-cognitive 
Strategies (strategy 
instruction) 

Evidence- 

based 

Interventions that address higher-order thinking, involving active 
control over cognitive processes. Strategies may include verbal 
self-instruction, self- evaluation and self-monitoring. 

0.69 

Perceptual-motor 

Programs 

Non 

evidence- 

based 

Improving academic performance by addressing perceptual and 
motor skills such as visual-motor abilities, physical coordination, 
balance activities and body awareness. 

0.08 

Problem-based 

Learning 

Non 

evidence- 

based 

Student centred learning occurs in small groups with a facilitator. 
Authentic problems are presented to develop required knowledge 
and problem solving skills. New information is acquired through 
self-directed learning. 

0.15 

Learning Styles 
Instruction 
(modality based) 

Non 

evidence- 

based 

This involves assessing the learner’s style of learning (i.e., visual, 
auditory, kinaesthetic) and matching instruction to the patterns of 
strength and weakness. For example, visual strategies might be 
emphasised with visual learners, auditory strategies might be 
emphasised with auditory learners and hands-on activities 
emphasised with kinaesthetic learners). 

0.41 

Peer Tutoring 

Evidence- 

based 

The systematic use of students to act as tutors for other students. 

0.55 

Setting Goals 

Evidence- 

based 

Clear goals are set for student performance. 

0.56 

Reading 

Comprehension 

Programs 

Evidence- 

based 

The use of activities to attempt to improve comprehension of 
written text. May include strategies such as asking questions 
during reading and summarising text. 

0.58 

Mastery Learning 

Evidence- 

based 

Mastery learning involves setting clear performance standards, 
regular testing of student learning as well as supplementary 
teaching of students who do not attain the required level of 
mastery. 

0.58 

Direct Instruction 

Evidence- 

based 

Structured instruction involving setting clear objectives, 
structured, clear and explicit teaching, regular checking of 
understanding, guided practice and independent practice. 

0.59 

Whole Language 
Reading 

Non 

evidence- 

based 

Reading instruction based on the concept that acquisition of 
reading skills is primarily dependent on context and words are 
learned more easily in the context of the words around them and 
the story. 

0.06 

Phonics instruction 

Evidence- 

based 

The systematic teaching of the alphabetic code (letter-sound 
correspondence) and how to use this knowledge to read words. 

0.60 

Formative 

Evaluation 

Evidence- 

based 

Systematic and regular testing of student performance (typically 
at least twice a week) while they are learning (as opposed to after 
completion of instruction). 

0.90 


Table 1: Instructional Strategies 


Note: ES - Cohen’s d effect sizes are reported by Hattie (2009 


Vol 40, 6, June 2015 


90 



Australian Journal of Teacher Education 


Procedures 

Following approval by the Human Research Ethics Committee, an information letter 
and consent form was forwarded to the Dean or relevant Head of Department of publicly 
funded Australian universities providing initial teacher education programs. In addition, the 
federal government Department of Education, Employment and Workplace Relations (n.d.) 
Commonwealth Register of Institutions and Courses for Overseas Students website was 
searched to assist in identifying additional approved private institutions that provided initial 
teacher education programs. A total of 48 institutions were identified as offering relevant 
programs. 

The information and consent letter consisted of a brief description of the research and 
institutional consent form. Consenting institutions were asked to distribute the link to all 
students in their final year of study via email, provide information on the number of final 
year students and provide a contact person regarding distribution. In the case of one 
institution, total possible participant numbers were estimated from graduation lists in the 
subsequent year. A reminder letter was sent to non-responding institutions four to eight 
weeks after the initial introductory letter. 

Once institutional approval was obtained, a recruitment notice was sent to students, 
providing the link to the survey along with a brief description of the research. Consistent 
with the ethics approval, students were also given the option of entering a draw for one of 
four incentive prizes on completion of the survey. Students were notified that the survey 
would close in four weeks. After two weeks, a reminder notice was forwarded to potential 
participants. 


Results 

A total of 15 (31%) of institutions consented to participate in the study and the total 
number of potential respondents was 3193 according to the information provided. Responses 
were received from 300 students but 10 of these were ineligible as they indicated they had 
more than 1 year left to complete their course of study, leaving a total of 290 responses and a 
response rate of 9.1%. Demographic characteristics of the sample are summarised in Table 2. 
In response to the question regarding level of program, four students selected the “other” 
option but, on examination, all of these responses could be reclassified into the alternative 
categories. Of the 23 respondents who selected other for the teaching area, 12 indicated that 
they were completing a combined early childhood and primary teaching program. Numerous 
secondary teaching areas were identified and many students identified multiple areas. Only 
teaching areas identified by more than 10 respondents are reported in Table 2. 

Information relating to Part 2 of the survey is provided in Table 3. Data are presented 
on the number of participants selecting each response option. In addition, the data were 
ranked across factors for each responding participant (n = 271) and the mean of each ranking 
across participants was calculated for each decision-making factor. Probably the most 
striking feature of these data was the number of very important ratings for practicum 
experience. Ordinal values from 1 (very important) to 5 (very unimportant) were assigned to 
ratings and unsure responses were treated as missing data for the purpose of analysis. A 
Friedman two-way analysis of variance (Strieker, 2008) was conducted to determine whether 
decision-making factors were ranked differently by participants. There were significant 
differences in the rankings, % 2 (5, N = 271) = 199.38, p < .0001, so Conover post hoc 
comparisons were completed. The results of these paired comparisons and mean ranking 
differences between interventions are presented in Table 4. All differences were significant 
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except for those between (a) empirical research and current teacher preparation, (b) teacher 
advice and research, and (c) teacher advice and teacher preparation. 

Data relating to ranking of the strength of evidence are presented in Table 5. Data are 
presented on the number of participants selecting each response option. In addition, the data 
were ranked across practices for each participant who provided a rating for all 14 practices 
(i.e., no “unsure” responses; n = 153) and the mean of each ranking across participants was 
calculated for each instructional practice. 

Ordinal values from 1 (very strong) to 5 (very weak) were assigned to ratings and 
“unsure” responses were treated as missing data for the purpose of analysis. A Friedman 
two-way analysis of variance (Strieker, 2008) was conducted to determine whether evidence 
for practices was ranked differently by participants. There were significant differences in the 
rankings, % 2 (13, N = 153) = 236.43, p < .0001, so Conover post hoc comparisons were 
completed. The results of these paired comparisons and mean ranking differences between 
interventions is presented in Table 6 . Formative evaluation was ranked significantly higher 
than two non-evidence-based practices and significantly lower than one non-evidence-based 
practice. Meta-cognitive strategy instruction was ranked significantly higher than two non¬ 
evidence-based practices. Phonics instruction for reading was rated significantly higher than 
one non-evidence-based practice but significantly lower than four others. Interestingly, 
phonics instruction was not ranked differently to whole language reading instruction. Direct 
instruction was ranked significantly higher than three non-evidence-based practices. Mastery 
learning was ranked higher than one non-evidence-based practice and significantly lower 
than the remaining five non-evidence-based practices. Reading comprehension programs and 
setting goals were both ranked significantly higher than three non-evidence-based practices. 
Finally, peer tutoring ranked significantly lower than five non-evidence-based practices and 
significantly higher than the remaining practice. A Pearson correlation was calculated 
between mean ranking for evidence and Hattie’s (2009) calculated effect size for the 13 
relevant practices. Since a lower mean ranking score indicates greater perceived evidence, a 
negative correlation reflects greater agreement. The calculated correlation was -0.31 (? = - 
1.05,p = 0.32). 

Data relating to ranking of proposed frequency of use of strategies are presented in 
Table 7. Data are presented on the number of participants selecting each response option. In 
addition, the data were ranked across practices for each participant who provided a frequency 
rating for all practices (i.e., no “unsure” responses; n = 159) and the mean of each ranking 
across participants was calculated for each instructional practice. 

Ordinal values from 1 (very frequently) to 5 (never) were assigned to ratings and 
“unsure” responses were treated as missing data for the purposed of analysis. A Pearson 
correlation was calculated between mean evidence and frequency ratings for each relevant 
practice (r = .93, t = 8.81 ,77 < 0.001). This indicated a very strong relationship between the 
rating of evidence and planned frequency of use of practices. 
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Characteristic 

n 

Gender 

Female 

217 

Male 

73 

Age (years) 

Median 

25 

Mean 

29 

SD 

8.8 

Level of Program 

Bachelor degree 

202 

Graduate Diploma 

61 

Masters 

23 

* Teaching Area 

Primary 

148 

Secondary 

141 

Special Education 

7 

Other 

23 

Secondary Teaching Areas 

One teaching area 

50 

More than one teaching area 

86 

**Secondary Teaching Areas (> 1 0 respondents) 

English 

35 

Science 

28 

Religion 

21 

PDHPE 

20 

History 

15 

Art 

14 

Mathematics 

14 

Time to Complete Qualification 

Less than 6 months 

266 

6 months to 1 year 

24 

First Qualification 

Yes 

168 

No 

122 

*** Previous qualifications 

Technical and Further Education (TAFE) 

7 

Bachelor Degree 

90 

Masters Degree 

7 

Doctoral Degree 

1 

Other 

2 


Table 2: Sample Demographics 


Note. * Responses allowed in multiple categories; ** Open-ended responses; *** Open-ended responses. 
Seven participants had both Master and Bachelor degree. 
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How important do you think it is to use instructional practices that 102 147 19 1 2 19 3.54 

have been shown by empirical research to be effective? In this 

context, the term empirical research involves testing effectiveness of 

practices using experiments in which student performance is 

measured. 

How important is it for a teacher to use instructional practices that 131 123 10 5 2 19 3.25 

accord with their personal philosophy and style of teaching? 

How important to you is the advice of other teachers in deciding 70 173 23 4 1 19 3.93 

which instructional practices you will use in the classroom? 

How important is research in deciding which instructional practices 73 154 32 9 3 19 4.02 

you will use in the classroom? 

How important is the content of your current teacher preparation 96 134 28 12 1 19 3.77 

course in deciding which instructional practices you will use in the 

classroom? 

How important is your practicum experience in deciding which 202 61 7 0 1 19 2.48 

instructional practices you will use in the classroom? 

Table 3: Descriptive Data on Importance of Factors in Instructional Decision-Making 
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Empirical 

Research 

Philosophy and 
Style 

Advice Teachers 

Research 

Current Teacher 
Preparation 

Philosophy and style 

0.29* 





Advice Teachers 

-0.39** 

- 0 . 68 *** 




Research 

-0.48*** 

-0 77 *** 

-0.09 



Current Teacher 
Preparation 

-0.23 

-0.52*** 

0.16 

0.25* 


Practicum Experience 

l Q 7 *** 

0 77 *** 

j 45*** 

1.55*** 

\ 29 *** 


Table 4: Friedman Post Hoc Analysis and Mean Rank Differences for Factors in Instructional Decision- 

Making 


Note. * p<0.05, ** p<0.01, *** pcO.OOl; positive values indicate higher ranking for factors listed in first 
column. 



Very Strong 

Strong 

Neutral 

Weak 

Very Weak 

Unsure 

Mean Rank 

Multiple Intelligences 

80 

101 

35 

10 

0 

9 

6.7 

Inquiry-Based Learning 

89 

113 

22 

3 

1 

7 

6.4 

Meta-Cognitive Strategies (strategy instruction) 

49 

104 

60 

10 

3 

9 

7.0 

Problem-Based Learning 

72 

121 

32 

2 

0 

8 

6.9 

Peer Tutoring 

45 

95 

71 

9 

1 

14 

9.0 

Perceptual Motor Programs 

25 

72 

68 

14 

5 

51 

10.0 

Setting Goals 

104 

100 

26 

1 

0 

4 

6.3 

Reading Comprehension Programs 

95 

90 

30 

4 

3 

13 

6.3 

Learning Styles Instruction 

82 

96 

40 

4 

2 

11 

7.3 

Mastery Learning 

41 

85 

69 

12 

2 

26 

9.2 

Direct Instruction 

105 

97 

22 

6 

0 

5 

6.2 

Whole Language Reading 

46 

97 

47 

9 

2 

34 

8.3 

Phonics Instruction 

57 

87 

42 

10 

4 

35 

8.3 

Formative Evaluation 

89 

83 

40 

7 

2 

14 

7.3 


Table 5: Descriptive Data on Ratings of Evidence for Instructional Practices 
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n 

o 

era 


o. 

r 


H 

c 

o 

3' 

era 


GO 

CD 


o 


c 

p 


^0 pc 


3 era 


v! 

r 


a 


o 

o' 


O" 

o 

cT 


Inquiry-Based 

Learning 

0.34 











Meta-Cognitive 

Strategies 

-0.27 

-0.61 










Problem-Based 

Learning 

-0.17 

-0.51 

0.11 









Peer Tutoring 

_2 3 *** 

-2.64*** 

-2 03*** 

-2 13*** 








Perceptual Motor 
Programs 

-3 28*** 

-3.62*** 

-3.00*** 

-3 11*** 

-0.98* 







Setting Goals 

0.37 

0.03 

0.64 

0.53 

2 67 *** 

3.64*** 






Reading 

Comprehension 

0.42 

0.08 

0.69 

0.58 

2 72 *** 

3 7Q*** 

0.05 





Programs 

Learning Styles 
Instruction 

-0.58 

-0.92* 

-0.31 

-0.42 

1 72*** 

2.69*** 

-0.95* 

- 1 . 00 ** 




Mastery Learning 

-2 51*** 

-2.85*** 

-2 24*** 

-2 34*** 

- 0.21 

0.77* 

-2 88 *** 

-2 93*** 

_1 92 *** 



Direct Instruction 

0.45 

0.11 

0.73 

0.62 

2 75 *** 

3 73 *** 

0.09 

0.04 

1.04** 

2.96*** 


Whole Language 
Reading 

Phonics Instruction 

-1.56*** 

-1 90*** 

_ [ 29 *** 

-1 40*** 

0.74 

\ 72*** 

-1 93 *** 

-1 98*** 

-0.98* 

0.95* 

-2 02 *** 

-1.63*** 

_ [ 97 *** 

-1.35*** 

-1 46*** 

0.67 

1.65*** 

-1.99*** 

-2.05*** 

-1.04** 

0 . 88 * 

-2.08*** 

Formative Evaluation 

-0.58 

-0.92* 

-0.30 

-0.41 

1 73 *** 

2 7Q*** 

-0.94* 

-0.99** 

0.01 

j 93 *** 

-1.03** 


Table 6: Friedman Post Hoc Analysis and Mean Rank Differences for Ratings for Research Evidence 


Note. * p<0.05, ** p<0.01, *** p<0.001; positive values indicate higher ranking for practices listed first column. 


-0.07 

0.99* 1.05** 
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Multiple Intelligences 
Inquiry-Based Learning 

Meta-Cognitive Strategies (strategy instruction) 

Problem-Based Learning 

Peer Tutoring 

Perceptual Motor Programs 

Setting Goals 

Reading Comprehension Programs 
Learning Styles Instruction 
Mastery Learning 
Direct Instruction 
Whole Language Reading 
Phonics Instruction 
Formative Evaluation 


54 

91 

66 

17 

1 

6 

7.3 

68 

108 

45 

8 

2 

4 

6.4 

49 

104 

60 

10 

3 

9 

7.3 

55 

97 

71 

7 

0 

5 

7.0 

32 

59 

101 

29 

7 

7 

9.2 

21 

42 

80 

43 

13 

36 

10.4 

118 

84 

27 

5 

0 

1 

5.5 

93 

86 

28 

13 

5 

10 

5.8 

64 

83 

62 

15 

3 

8 

7.3 

24 

67 

91 

27 

2 

24 

9.4 

96 

96 

35 

6 

1 

1 

5.9 

50 

67 

63 

19 

9 

27 

8.0 

43 

68 

51 

22 

22 

29 

8.6 

65 

95 

47 

17 

2 

9 

7.0 


Table 7: Descriptive Data on Ratings of Frequency for Instructional Practices 


Discussion 


Instructional Decision-Making 

In interpreting data in instructional decision-making, it should be noted that most 
factors were considered “very important” or “important” by the vast majority of participants, 
with distribution between these two ratings determining rankings. Nevertheless, practicum 
experience was highest ranked by a substantial margin and was significantly higher than all 
other factors. Accommodating personal philosophy and style was second ranked, significantly 
higher than all other factors except practicum. The third highest ranked factor was empirical 
research, which was significantly higher than teacher preparation programs, advice from other 
teachers and research in general. Although it is arguably encouraging to see empirical 
evidence ranked higher than such factors as teacher advice, it was still ranked lower than 
practicum experience and personal philosophy and style. Camine (2000, p. 9) has argued that 
a “ mature profession ... is characterized by a shift from judgments of individual experts to 
judgments constrained by quantified data that can be inspected by a broad audience, less 
emphasis on personal trust and more on objectivity”. By this standard, the present data would 
suggest that education might have some way to travel before becoming a mature profession. 
The data presented in the current study indicates that teachers may be continuing to place 
greater weighting on personal experience and preferences than evidence. 

Interestingly, empirical evidence was rated higher than teacher advice in the current 
study yet Landrum et al. (2002) reported that late-stage trainee teachers viewed advice as 
more accessible, usable and trustworthy than professional journals. This apparent discrepancy 
might be accounted for by the exclusive focus of Landrum et al. (2002) on the source of the 
information (i.e., professional journals), rather than the nature of the content (empirical 
research). It was also of note that empirical research was specifically rated higher than 
research in general in the present study. This may suggest some level of recognition of the 
importance of such research. 
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The pre-service teachers in the current study viewed practicum as the most important 
influence on their instructional decision-making. The important role of practicum experience 
in shaping student teacher pedagogical understanding has been subject to exploration (e.g., 
Bronkhorst, Koster, Meijer, Woldman, & Vermunt, 2014; Walton & Rusznyak, 2013). Tetley 
and Jones (2014) found that exposure to various types of literacy programs during practicum 
was associated with student teachers’ knowledge of scientifically-based language concepts 
related to reading, providing some corroborative evidence for the perceptive importance of 
practicum reported in the current study. 

In summary, data from the present study would tend to indicate that there is a 
considerable way to go in attempting to bridge the research to practice gap in education. 
Although empirical evidence related to student outcomes was considered somewhat important 
in trainee teacher decision-making, it was still rated as lower than personal preference related 
to teaching style and much lower than practicum experience. These findings have a number of 
implications. In the longer term, a greater focus on the role of empirical research might be 
needed in teacher preparation programs. In the short-term, given the reported prominence of 
practicum experiences in decision-making, the importance of high quality and evidence- 
informed mentorship during practicum is highlighted. 


Research Evidence and Anticipated use of Practices 

Overall, data from teacher ratings of the strength of evidence provided a positive 
association between mean ranking of evidence and Hattie’s (2009) calculated effect sizes but 
the correlation was only modest. In contrast, mean ranking for evidence and mean ranking for 
intended frequency of use were highly correlated (r = 0.93) indicating that students intended 
to use strategies they judged as best supported by evidence. In contrast with previous 
research, students were offered the option of indicating that they were “unsure” in response to 
evidence (and frequency) rating. It seemed reasonable to expect students might be unfamiliar 
with some areas (e.g., approaches to reading instruction for many secondary teachers) and 
forcing a rating in such circumstances seemed inappropriate. Given this, the very high level of 
confidence of respondents was of particular note. The highest level of “unsure” responses was 
for perceptual motor programs (22%) with only three other practices exceeding 10% (phonics 
instruction, 10%; whole language reading, 14%; mastery learning, 11%). Bain et al. (2009) 
reported that pre-service teachers tended to endorse described interventions despite 
acknowledged lack of prior knowledge or exposure. Their suggestion that “more attention 
should be paid to teaching critical evaluation skills as a part of preliminary training of future 
educators” (p. 71) would appear to be supported by the current study. 

It was noticeable that across all instructional practices, very weak and weak ratings 
were infrequently used. In addition, for all practices, combined very strong and strong ratings 
exceeded neutral ratings, in most cases by a very wide margin. The belief that instructional 
practices were effective extended to practices that have consistently weak research support 
such as perceptual motor programs (Hyatt, Stephenson, & Carter, 2009; Kavale & Mattson, 
1983) and modality-based learning styles instruction (Kavale & Fomess, 1987; Pashler et al., 
2008; Landrum & McDuffie, 2010). 

Thus, teachers in the final stages of preparation tended to judge instructional practices 
as evidence-based, regardless of the actual evidence on effectiveness, and rating of strength 
correlated strongly with intended frequency of use. These data are not inconsistent with the 
findings of some other lines of research. For example, Bain et al. (2009) found that 
endorsement rates by pre-service teachers of evidence and non-evidenced-based practices, 
related to children with disabilities, did not vary in a consistent or logical manner. Similarly, 
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studies of reported frequency of use of instructional practices of graduate special education 
teachers in a number of countries (Burns & Ysseldyke, 2009; Carter et al., 2011; Carter et al., 
2012 ) have indicated that both evidence-based and non-evidence-based practices are 
simultaneously claimed to be used at moderate to high levels. 

The possible reasons for the lack of differentiation in judgement regarding the 
empirical evidence-based for the instructional practices examined in the present study is open 
to speculation. It is possible that the diversification of curriculum demands, particularly for 
primary teachers, may mean that limited attention is being directed to examining efficacy 
research. It is also possible that teacher preparation programs may direct student attention to 
more effective techniques but, as a result of time constraints, fail to adequately highlight 
procedures that have weak evidence (Carter et al., 2011). It has also been suggested that 
teacher preparation programs may not adequately provide teachers with the tools to make 
judgements regarding evidence-based interventions (Everett et al., 2008; Jones, 2009). In 
addition, noting that much of the pressure for a move to empirical evidence-based practices 
has come from sources external to the profession, it is possible that the level of commitment 
to evidence-based practice within the teacher preparation sector may be more circumspect. 
Finally, teaching is perhaps unique as a profession as prospective teachers typically have 
more than a decade of exposure to teaching practices prior to any formal professional training 
(Everett et al., 2008). Teachers in preparation may tend to interpret evidence through the lens 
of their own experience as students rather than examining empirical research evidence. The 
present study was not designed to offer insights into this issue but this should be considered a 
priority for future research. 


Limitations 

A limitation of the present research was the low response rate. Only 31% of 
institutions consented to participate in the research. In a small number of cases, explanations 
were offered such as the large number of research studies in which students were invited to 
participate or that students would be unlikely to be monitoring student email due to breaks 
and practicum. Nevertheless, the low participation rate was surprising given the limited 
demands placed on institutions. 

In addition, the response rate from students was only around 10%. O’Neill and 
Stephenson (2012b), who also recruited through email invitations, reported a comparable 
return rate of 14.2%. Response rates from internet surveys can be low (Monroe & Adams, 
2012 ) and, despite strategies to maximize responses such as a follow-up letter and incentive 
prize, this was the case in the present study. It was not possible to determine how actively 
students monitored their student email addresses and, consequently, how many received the 
invitations. Thus, as is often the case in survey research, caution must be exercised in 
interpreting results due to possible bias in participant selection. It should also be noted that 
data regarding implementation was based on reported intent and it is not known how this may 
correspond to actual implementation. 

Finally, Hattie’s (2009) synthesis of meta-analyses was used for evaluation of the 
strength of evidence for instructional practices. This is a somewhat blunt measure and Hattie 
has noted that his intent was to provide an “explanatory story” and weave practices into a 
coherent narrative to give general flavour of the types of instruction that are more likely to be 
effective. As such, and given the often poor quality of educational research (Everett et al., 
2008), conclusions regarding specific educational practices should be treated with a degree of 
caution. Although it can be argued that Hattie gives a reasonable broad brushstroke overview 
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of our current state of knowledge, this is certainly not definitive and may change as more and 
higher quality data become available. 


Conclusion 

The present study has suggested that while students believe empirical evidence is 
important in selection of instructional practices, personal preference and in particular 
practicum experience are considered more important factors. Given the reported prominence 
of practicum experiences in decision-making, the importance of high quality and evidence- 
informed mentorship during practicum is highlighted. Final year student teachers tended to 
rate most practices as having a strong or very strong evidence base. Relatively few 
respondents indicated that they were unsure regarding the level of evidence for practices, 
suggesting a high level of confidence in their knowledge. Nevertheless, there was only a 
modest correlation between mean student evidence ranking and Hattie’s (2009) effect sizes. 
Further, absolute ratings of evidence strength often did not correspond well with available 
empirical evidence. The difficulty that students encountered in differentiating evidence-based 
from non-evidence-based practices suggests that further progress needs to be made if 
education wishes to become a truly evidence-based profession. 
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